Photogrammetry is a computational technique in which a plurality of images (such as stereo images) can be used to determine the location of points captured in the images relative to the position of the sensor that captured the images. To perform such analysis, however, the multiple images must first be oriented with respect to one another; such orientation defines the spatial and angular relationships between the two scenes in the captured images (or, conversely, the spatial and angular relationships between the sensors when capturing the respective images).
One technique for orienting a plurality of images (such as stereo images) is termed, “bundle adjustment,” which is the technique of simultaneously refining the three-dimensional coordinates describing the scene geometry of each captured image, and/or relative orientations of the image capture system(s) that captured the images. Essentially, bundle adjustment attempts to optimize (minimize) the reprojection error between the image locations of observed and predicted image points; this error can be expressed as the sum of squares of a large number of nonlinear, real-valued functions.
Accordingly, bundle adjustment is computationally expensive, because it requires first the identification of conjugate points in the several images (i.e., grid locations in each image of the same real-world feature captured in all of the images) and then the optimization of the equations expressing the positions of those conjugate points in each image. The step of identifying these conjugate points (termed “conjugate matching”) is especially compute intensive because it involves identification of similarities in groups of adjacent pixels in each of the image, which requires comparison of huge numbers of pixels in each image against one another to identify conjugate candidates, which can then be analyzed to determine whether the respective points depicted by the pixels in each image actually represent conjugate points.
Currently the methods for conjugate point matching on stereo images can be divided into two groups: area based matching and feature based matching. The methods of area based matching require very accurate initial position of conjugate candidates. The methods of feature based matching also need a general area of the potential candidates. Although the employment of epipolar geometry can reduce the search scope for correct conjugate point from two dimensions to one dimension, the one dimensional search range is still very large for large imagery data. For instance, the size of a single image from a typical camera is 2592 by 1944. Without a pre-determined search range, the method has to go through each pixel on the epipolar line on one of the images to identify a point on the other image. Therefore, to find the conjugate points from one image capture position/orientation (i.e., a “station”) to another station is a tremendously processing-intensive task involving comparison along the entire epipolar line.
To alleviate this problem, one might attempt to reduce the search range (i.e., the portion of an image that is searched for conjugate matches) arbitrarily. An incorrect search range, however, results in an incorrect or failed match, while a large search range may increase the both possibility of a wrong match (due to the large number of potential conjugate candidates) as well as computation time.
Currently-known feature based methods always avoid the problem of determining the search range, and directly begin with an assumption that the area of conjugate points is known, or at least roughly known. The area based methods are usually applied in conjugate point matching on aerial or satellite images in which, the overlap between stereo images, flight height of aircraft, and general terrain configuration are available, which allows relatively inexpensive computation of the position of correct conjugate points can be accurately.
Neither of these types of methods, however, can efficiently handle conjugate matching on panoramic images taken at ground level. For example, provisional U.S. Patent App. No. 61/710,486, filed Oct. 5, 2012 by Grässer et al. and titled “Enhanced Position Measurement Systems and Methods,” as well as U.S. patent application Ser. No. 13/332,648, filed Dec. 21, 2011 by Grässer et al. and titled “Enhanced Position Measurement Systems and Methods” (both of which are hereby incorporated herein by reference and which are collectively referred to herein as the “Incorporated Applications”) describe position measurement systems that feature a suite of panoramic cameras that provide, in some cases, up to 360° photographic coverage around a capture point.
Unlike the targets in aerial or satellite images, which are approximately on the ground plane and have a predictable distance to the camera in the aircraft, the distance from targets in panoramic ground images, such as building facades, captured by such a ground-based position measurement system can vary according to different targets. Consequently, it is almost impossible to predict the position of the conjugate points in the stereo images. Therefore, the traditional point matching methods, area based or feature based, usually fail in conjugate point matching for such panoramic images, because the search algorithm has no clue about where to search for the correct match.
Accordingly, there is a need for more robust point matching techniques. It would be particularly helpful if such techniques could be employed in the context of panoramic stereo images.